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ABSTRACT 

Qualitative evaluation and quantitative evaluation 
are discussed, examining how the perspective of each can and should 
create a critical debate that encourages inquiry instead of 
hostility. To achieve the major goal or linking evaluation 
information to the organization's culture, the evaluator needs to use 
multiple methods in a responsive, issue-raising, contextually 
sensitive approach. While some evaluators and researchers propose 
that the qualitative and quantitative approaches cannot be 
synthesized because of their polar epistemological stances, others 
develop compatibility of methods as their main focus. Arguments that 
the incompatibility thesis presents cannot be dismissed. However, 
discrepancies should prompt the researcher to probe particular issues 
in greater depth. This process may expand knowledge about evaluation 
in terms of increasing the likelihood of audi ences engaging in thei r 
own knowledge expansion. In the practice of evaluation, the evaluator 
should be aware of the various qualitative and quantitative methods 
to make the evaluation a process of identifying, educating, and 
communicating the perspectives and concerns of the evaluator and the 
stakeholders to maximize the relevance and success of the evaluation 
and the program. (SLD) 
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SYNTHESIZED APPROACHES: EXPANDING THE PERSPECTIVES AND IMPACT OF 
QUALITATIVE AND QUANTITATIVE EVALUATION' 

Mary E. Piontek 

When I first envisioned this paper, I reflected on my limited evaluation 
background and looked for sustenance from my wider experience in literature 
and literary criticism. In integrating my eclectic background, I focused on 
the main theme of this year's conference, combining or synthesizing the 
perspectives of qualitative and quantitative philosophies and practices, which 
is a concern emphasized in literary criticism, in addition to evaluation. I 
wanted to discuss an approach that enhanced the significance of evaluation 
information to stakeholders/clients, increased the utilization and impact of 
evaluation results, and expanded the possible applications of evaluation. 

However, before I could even begin the process of focusing on how the two 
methods/practices can work in a form to increase the significance of 
evaluation information to the stakeholders/clients, I had to make my way 
through the two basic stances surrounding the quantitative/qualitative debate. 
The incompatibility thesis focuses on the paradigmatic conflicts between 
quantitative and qualitative research and the inherent link between methods 
and philosophy. The proponents of the compatibility thesis support the 
combining of methods because they propose that the link between practice and 
paradigm is tenuous and that evaluation problems/questions and audiences 
necessitate the use of combined methods. 

In this paper, I will not attempt to resolve this conflict, but rather 
briefly discuss each stance and look at how understanding the perspective of 
each can and should create critical debate that encourages inquiry instead of 
hostility. I will also suggest that the debate or, perhaps a better term, the 
discussion should continue. There is no need to consider the conflict resolved 
by either group. The link between paradigms and methods needs to be questioned 
and revised as we come to understand our values and "truths" about our 
perspectives. The existence of alternative and diverse viewpoints need not be 
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a hostile clash of intolerant rhetoric; the diverse critical beliefs can drive 
us towards the common purpose of improving the field of evaluation. Open 
discussion can enhance the significance of evaluation information to 
stakeholders/clients, increase the utilization and impact of evaluation 
findings, and expand possible applications of evaluation. Without tolerant 
discussion we will alienate each other - as well as stakeholders/clients, 
stagnate the field, and make the development of new theories and more fruitful 
areas of inquiry impossible. 

A critical approach should create meaning and understanding for its 
audiences (s) by revealing information; framing context; organizing patterns, 
signs, signals, and data; expanding boundaries; developing new language and 
voices; juxtaposing ideas/images; and by revealing social/structural 
interaction and cultural significance. Both qualitative and quantitative 
research and their corresponding epistemological philosophies should enable 
the evaluator/ researcher to do this. 

My conceptualization of evaluation is that it is a process of systematic, 
collaborative inquiry that uncovers or discovers information that is open to 
interpretation, critique, and analysis, within varied, political, ever 
changing contexts. Evaluation may determine value, change, or improvement 
within the program or situation being evaluated and may directly or indirectly 
affect the decision-making process of the organization. 

The goal(s) of an evaluation can encompass many aspects of a given program 
or situation: determining its worth and/or value, providing information for 
decision-making, facilitating conceptual and/or instrumental use, or educating 
clients about evaluation. However, unless an evaluator is able to politically 
and contextually analyze the setting or situation in which the progrcim and 
evaluation occurs, none of these secondary goals will effectively take place. 

Documentation of the context of an evaluation or organization can provide 
valuable information about its operation to its constituents, thereby helping 
them to develop their conceptualization of the program or organization and 
communicate assumptions, biases, and developing perspectives of both the 
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evaluator and the constituents (Torres, 1988). By analyzing, describing, and 
conununicating information about the evaluation environment, the evaluator 
helps facilitate understanding of and action about the evaluation, ita 
purposes, and its results: in essence, 'making sense' of the concerns of 
decision-makers, the progrcun goals, the conflicts between various 
stakeholders, and the political nature of a program and/or evaluation (Torres, 
1988, 1991). 

In order to achieve this major goal of linking evaluation information to 
the organization's culture, the evaluator needs to employ "multiple methods in 
a responsive, issue raising, contextually sensitive approach" (Torres, 1991, 
p. 196) . 

Some evaluators/researchers propose that the two approaches of qualitative 
and quantitative cannot be synthesized because of their polar epistemological 
stances. Others do not view the epistemological level as the determinant of 
method-choice and thus develop compatibility of methods as their main focus. 

INCOMPATIBILITY 

The incompatibility thesis focuses on the paradigm conflicts between 
quantitative anJ qualitative research and insists that there is an inherent 
link between methods and philosophy. Dan Bednarz, 1985, insists that the two 
approaches cannot be synthesized. He suggests ^hat current discussions of 
synthesis overlook, ignore, or misinterpret the incompatibilities of the two 
approaches. The two approaches differ philosophically in terms of conceptions 
of data, validity, reliability, and causality, as well in the conception of 
social science's mission (Bednarz, 1985). He suggests that a synthesis of the 
two approaches would necessitate the adoption of one perspective over the 
other; it is logically impossible for there to be a true middle ground 
(Bednarz, 1985). The incompatibility thesis is further supported by John K. 
Smith and Lous Heshusius who suggest that the compatibility claim cannot be 
sustained for it ignores paradigmatic differences and confuses two definitions 
of 'method'. For them inquiry is not and should not be "what works" at the 
level of individual research procedures, but must be a clear conceptualization 



of incjuiry at the paradigmatic level (1986, p. 8). 

Th"' proponents of the incompatibility thesis suggest that the literature 
that supports the integration of perspectives overlooks the subtleties of the 
approaches, inappropriately interprets one perspective in terms of the other 
(most commonly qualitative in terms of quantitative) , and inappropriately 
assumes that the two philosophies/epistemologies are stable in nature 
(Bednarz, 1985). 

COMPATIBILITY 

The compatibility thesis supports the combining of methods. Proponents 
present the link between practice and paradigm (qualitative/ interpretive, 
quantitative/positivism) as tenuous (Addis, 1987; Brannen, 1992; Bryman, 1992; 
Brewer and Hunter, 1989; Fetterman, 1989; Firestone, 1986; Hammersley, 1992; 
Hillocks, 1992; Howe, 1988; Reichardt and Cook, 1979; Weinholtz et. al . , 
1992). However, there are many different thoughts among these propcnents as to 
why compatibility or synthesis may be possible, A few perspectives are 
discussed hare. 

Gregg Addis supports the combined use of qualitative and cjuantitative 
methods in educational research stating that "educational research must not 
pursue a monolithic form of research in which only qualitative or cjuantitat ive 
research is performed" (1987, p, 13), In discussing Morgan 1983, he points out 
that each form offers insight that the other cannot generate. He suggests that 
educational researchers must focus on the research problem to determine the 
optimal design, which may or may not include both methods. The design should 
reflect the" particular and unique reality" of the problem (Addis, 1987, pp. 
21-2). He suggests that the debate over the philosophical differences should 
perhaps be seen as "no mutual exclusions between Idealism and Realism. Maybe 
both systems of thought are 'true' descriptions of reality .. .Perhaps reality 
is subjective: Objects are a reflection of the mind, and reality is objective: 
objects are distinct and independent from the mind" (Addis, 1987, pp. 20-21). 

Martyn Hammersley argues that the distinction between qualitative and 
quantitative is of limited use for it does not capture the full range of 
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options for inquiry. The diversity of social research cannot be encompassed in 
many paradigms, let alone within two* In discussing seven issues of social 
inquiry, Hamraersley suggests that it is not a simple contrast between two 
opposed standpoints, but "a range of positions sometimes located on more than 
one dimension" (1992, p» 51) ♦ 

He further believes that "there is no necessary relationship between 
adopting a particular position on one issue and specific positions on another • 
Many combinations are quite reasonable* the purposes and circumstances of 
research should be determinants of selection among positions rather than being 
derived from methodological or philosophical commitments" (Hammersley, 1992, p* 
51). 

Similarly, David Fetterman (1989) uses an university evaluation to 
illustrate the advantages of a mix of qualitative and quantitative approaches* 
He suggests that combining the quantitative and qualitative approaches 
enhances the usefulness and accuracy of the evaluation by more clearly 
illuminating the process and outcome of a study* He doubts that a study can 
be conducted well without the combined use of both methods* Because of their 
educational and evaluation experiences, evaluators most often have a 
predisposition toward the use of one method-type* However, as Fetterman states 
"the evaluator's predisposition and the dominance of an approach in a given 
study does not suggest exclusivity, lack of rigor, or the complete absence of 
any other approach" (1989, p* 1)* Instead of focusing on the philosophy or 
"the intrinsic value" of any approach, the nature of the research/evaluation 
problem and the audience should determine the best way to collect and present 
findings (Fetterman, 1989)* 

The combination of the two methods can enhance the utility and accuracy of 
the evaluation by capturing an accurate portrayal of events and participants 
(Brannen, 1992; Bryman, 1992; Eisner, 1991; Fetterman, 1989; Firestone, 1986; 
Hillocks, 1992; Kidder and Fine, 1987; Reichardt and Cook, 1979; Smith, M*L*, 
1986; Sprague and Zimmerman, 1989; Wallen, 1989)* For instance^ the 
combination of methods can enhance the impact of an university evaluation 
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(Petterman, 1989), as well as strengthen the external validity in the 
licensing of health care professionals (Gothler, 1992). Qualitative data can 
help to illustrate the soroetiroes ambiguous or misleading results of 
quantitative studies and help develop connections with previous research, as 
well as provide direction for future research (Weinholtz, 1992 )• 

In his discussion of reconciling the qualitative and quantitative in 
literary research, George Hillocks, Jr. states that the extreme polarities are 
not a viable position. Quantitative researchers use interpretive methods in 
problem finding, in the explanation of the relationship of data to claims, in 
theory building, and in explaining particular cases in connection with 
established knowledge and theory (Hillocks, 1992). Both the qualitative and 
quantitative researcher needs to explain procedures, verify observations, and 
oross-check sources. Their methods are quite similar for generalizing/sample 
size considerations and each approach prediction - just in a different way 
(Hillocks, 1992). 

Hillocks does point out that the two types of research can and do focus on 
different kinds of problems; however he does suggest that the findings and 
claims can be synthesized through the use of ordinary argument: 



However the methods are more than complementary in the 
general sense of enabling researchers to deal with 
different kinds of problems [and]... no single 
conception of science is necessary for differi.ng 
research methodologies to be used successfully in a 
complementary fashion. . .although the methods used to 
describe claims may be in conflict, the claims 
themselves are not - or at least not necessarily sc.. 
claims derived from quite different research methods 
can be used in a complimentary way to establish or 
disestablish propositions and theories (1992, pp. 61- 



He goes on to suggest that in literacy research (and I would argue in 
evaluation as well) it is important to establish what appears to be objective 
and interpret these findings to give them meaning and significance, to do so 
necessitates the use of both sets of assumptions or methods. Hillocks asks us 
"Is it not possible that our divergent assumptions about objective r ality 
simply represent different metaphors about our relationship to reality, and 
that both have validity under certain sets of conditions?" ( 1992, p. 64). 
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William A. Firestone argues that the connection between method and research 
paradigm ia not so much logical as it is rhetorical (1986) • The language used 
exprekises the assumptions of the phenomenological paradigm. While rhetorically 
different the results of the two methodologies can be complementary; there is 
an aesthetic connection between method and paradigm (Firestone, 1986)* 

The purist and the pragmatist each has her different view and uses the 
corresponding quantitative or qualitative rhetorical devices to present " a 
different view of the phenomenon studied and uses different means to persuade 
the reader of the validity of the conclusions drawn" (Firestone, 1986, p. 1). 
"Yet^ the are not antithetical. They can present the reader with different 
kind of information and can be used to triangulate to gain greater confidence 
in one's own conclusions" (Firestone, 1986, p. 1). 

Scientific writing is standardized, bare, and limited in rhetorical excess* 
Its apparent absence of style is in effect a rhetorical device in itself. It 
has persuasive power in that it tries to convince the reader of the 
objectivity and singularity of meaning. Elaboration builds reference to some 
larger meaning, rich in multiplicity of meaning (Firestone, 1986). While 
quantitative and qualitative studies use different strategies to persuade 
readers of their validity and provide different assumptions about phenomena, 
they also provide complementary information to the readers; quantitative 
dev.. OS the magnitude of relationships, and qualitative provides concrete 
description of detail, active process, and audiences' perspectives (Firestone 
1986) . 

However, this does not suggest that evaluators or researchers do not have 

philosophical or epistemological beliefs behind choosing a particular method 

or that the methods are completely interchangeable. As Firestone states 

these rhetorical differences in method do not negate 
the view that there are instrumental reasons for 
choosing particular methods in specific 
situations. . Each method type does provide different 
kinds of information. Their strengths and weaknesses 
are complemenv.ary . . . Thus , each can make a contribution 
to a reader who is concerned about the subject of 
study without being firmly committed to either 
paradigm. ♦ .while there are a number of reasons for 
selecting a met>od':,; n-^^-ical approach, one's decision 



often expresses values about what the world is like, 
how one ought to understand it, and what the most 
important threats to understanding are (1986, p. 16). 

In practice it is unusual for philosophy or epistemology to be the only 
determinant of methods. Research projects are influenced, but not 
automatically determined by philosophy (Brannen, 1992). Julia Brannen 
questions "as to how far epistemological issues in practice determine 
methods. .the converse may also be posed, namely whether the use of a 
particular method inevitably means that a particular epistemological position 
has been adopted. . there is no necessary or one-to-one correspondence between 
epistemology and methods" { 1992 , p. 15). 

CONTINUING THE DISCUSSION 

Although I do not support the incompatibility thesis, I importantly do not 
dismiss the arguments it presents. Much of the literature focusing on 
synthesis does fail to address the epistemological differences between the 
qualitative and quantitative paradigms and only discusses combining the two 
approaches at the application of individual procedures. (Although here I have 
tried to choose examples of compatibility discussants who do discuss 
epistemological issues.) Interestingly, documenting the compatibility of 
methods at the application level can drive discussion of epistemological 
positions and their influences on social enquiry (Brannen, 1992). 

We seek to understand society at both the macro- and micro- level, yet one 
research method cannot address both levels of inquiry. "In so far as the 
findings which result from different methods are at odds or conflict with one 
another, these seeming contradictions ought to be addressed by the researchers 
in their interpretation of the data and the linkages they make between 
methods, data, and theory. Discrepancies should also prompt the researcher to 
probe particular issues in greater depth. In their own right, these may lead 
to new theories and more fruitful areas of enquiry" (Brannen, 1992, p. 17). 
This process may expand knowledge about evaluation in terms of increasing the 
likelihood of audiences engaging in their own knowledge expansion through 
different ways of knowing or approaching qualitative and quantitative 
information. 

We cannot close off the possibility of creative generation of new theories 
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or research avenues because of hostile and unproductive debates. We must try 
to address our seeming contradictions and in doing so begin to develop a 
language and writing style that can encompass these linkages. The 
compatibilist view admits that social research needs to use a vocabulary 
appropriate for describing social events, "which means using intent ionalist 
( 'ontologically qualitative') concepts"; however quantification is also needed 
and necessary to understand society (Howe, 1988, p, 14), The use of Giddens ' 
notion of the "double hermeneutic" in a working back and forth between "the 
technical, scientific vocabulary of social science and the wordaday natural 
vocabulary of social conduct" which moves from specific/local/near detail to 
global/distant structure may be a starting point for such language development 
(Howe, 1988, p. 14). 

In the practice of evaluation^ the evaluator should be aware of the various 
qualitative and quantitative methods that best elicit informative relevant to 
the evaluation. The evaluation should be a process of identifying, educating, 
and communicating the perspectives and concerns of both the evaluator and the 
stakeholders to maximize the relevancy and success of the evaluation and the 
program, 

John K, Smith and Lous Heshusius, proponents of the incompatibility thesis, 
believe that the "conversation" or discussion needs to be kept open in order 
to more fully discuss the issue of objectivism versus realism (1986), This 
call to keep the conversation open is indeed echoed by proponents of combining 
the two methods as well. Debate should and must continue about the relative 
usefulness of different methods to study specific problems/i^ituations 
(Reichardt and Cook, 1979), 

"One of the central concerns behind incompatibilism is legitimate: To the 
extent that preoccupation with methods prompts unref lectiveness and stifles 
progress, educational researchers ought to be pressed to take a look at deeper 
epistemological issues" (Howe, 1988, p. 13-14) However, this should not eq^ual a 
forced choice between the two paradigms, 

Diane Stephens and P, David Pearson, in discussing literary criticism and 
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perspectives^ suggest that by understanding an opposing or, as I prefer a less 
hostile term, alternative critical perspective we gain a fuller and richer 
understanding of our own perspective through critical reflection (1992). By 
becoming tolerant and appreciative of the knowledge engendered by diverse 
perspectives, we can move the field, as well as ourselves and others - 
especially stakeholders and clients, towards a better understanding of 
evaluation* What Stephens and Pearson propose for literary criticism should 
also be heeded by evaluators; by understanding alternative viewpoints we 
improve our ability to use multiple methods, we come to share a common 
interest in improving the field, and we ensure and perpetuate the need for 
dialogue and creative research (1992). The tension between the paradigms 
should drive inquiry rather than "reify sides in hostile debates" (1992, p. 
354) . "Tolerance must be the ethical principle underlying all our work. 
Tolerance permits us to recognize, respect, learn from, and celebrate the 
diversity of perspectives that our literary [and I would add evaluation] 
community offers" (Stephens and Pearson, 1992, p. 350). 
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